Monitoring system and monitoring method

ABSTRACT

A monitoring system includes a memory, and a processor coupled to the memory and configured to detect a face area of a subject from a captured image divided into a plurality of blocks according to a number of light emitting elements that irradiate an image capturing range, and control a light emission intensity of a light emitting element corresponding to a block including the face area among the plurality of blocks according to one of a position of the face area and a size of the face area.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of the priorJapanese Patent Application No. 2019-219049, filed on Dec. 3, 2019, theentire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein are related to a monitoring system anda monitoring method.

BACKGROUND

As an example of a biometric authentication technology based onbiometric images, a face authentication technique using a monitoringcamera installed outside a building or at a building entrance is knownin the related art. In the face authentication technique, in order tosuppress the influence of backlight or oblique light, for example, anillumination device that irradiates a living body is attached to themonitoring camera to improve the image quality of a biometric image tobe captured.

Related techniques are disclosed in, for example, InternationalPublication Pamphlet No. WO 2016/084214.

SUMMARY

According to an aspect of the embodiments, a monitoring system includesa memory, and a processor coupled to the memory and configured to detecta face area of a subject from a captured image divided into a pluralityof blocks according to a number of light emitting elements thatirradiate an image capturing range, and control a light emissionintensity of a light emitting element corresponding to a block includingthe face area among the plurality of blocks according to one of aposition of the face area and a size of the face area.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view illustrating an example of the system configuration ofa monitoring system;

FIGS. 2A to 2C are views for explaining the configuration of amonitoring device;

FIGS. 3A and 3B are views illustrating a configuration example of adiffractive optical element;

FIG. 4 is a view illustrating an example of the hardware configurationof a control device;

FIG. 5 is a first view illustrating an example of the functionalconfiguration of the control device;

FIGS. 6A and 6B are first views illustrating an operation example of themonitoring system;

FIG. 7 is a first flow chart illustrating a flow of control process bythe control device;

FIG. 8 is a second view illustrating an example of the functionalconfiguration of the control device;

FIGS. 9A and 9B are second views illustrating an operation example ofthe monitoring system;

FIG. 10 is a second flowchart illustrating a flow of control process bythe control device;

FIG. 11 is a third view illustrating an example of the functionalconfiguration of the control device;

FIG. 12 is a third flow chart illustrating a flow of control process bythe control device;

FIG. 13 is a fourth view illustrating an example of the functionalconfiguration of the control device;

FIGS. 14A and 14B are third views illustrating an operation example ofthe monitoring system; and

FIG. 15 is a fourth flowchart illustrating a flow of control process bythe control device.

DESCRIPTION OF EMBODIMENTS

Since the brightness of an illumination light decreases in inverseproportion to the square of a distance, the face of a distant subjectappears dark while the face of a nearby subject appears bright. That is,the image capturing distance at which an image may be captured withappropriate brightness is limited.

Hereinafter, embodiments of a technique capable of capturing a subject'sface with constant brightness will be described with reference to theaccompanying drawings. Throughout the present disclosure and thedrawings, elements having substantially the same functionalconfiguration are denoted by the same reference numerals, andexplanation thereof will not be repeated.

First Embodiment

System Configuration of Monitoring System

First, the system configuration of a monitoring system will bedescribed. FIG. 1 is a view illustrating an example of the systemconfiguration of a monitoring system. As illustrated in FIG. 1 , amonitoring system 100 includes a monitoring device 110, a control device120, and an authentication device 130.

The monitoring device 110 is a monitoring camera with so-called spotlighting. Specifically, the monitoring device 110 has an image capturingunit 111. The monitoring device 110 also includes a plurality of sets ofLEDs (Light Emitting Diodes) which are an example of light emittingelements, and diffractive optical elements. The example of FIG. 1represents a case where the monitoring device 110 includes 12 sets ofLEDs and diffractive optical elements (see sets 112_1 to 112_12).

The image capturing unit 111 captures an image of a subject (notillustrated) and transmits the captured image to the control device 120.Each of the sets 112_1 to 112_12 of LEDs and diffractive opticalelements operates independently based on an instruction from the controldevice 120, and irradiates an image capturing range of the imagecapturing unit 111 with spot lighting. The light emission intensity ofeach of the sets 112_1 to 112_12 of LEDs and diffractive opticalelements is controlled based on an instruction from the control device120.

A control program is installed in the control device 120, and byexecuting the control program, the control device 120 functions as animage processing unit 121 and an illumination control unit 122.

The image processing unit 121 detects a face area of the subject fromthe captured image that has been received from the image capturing unit111. The image processing unit 121 also determines the light emissionintensity of each of the sets 112_1 to 112_12 of LEDs and diffractiveoptical elements, based on the detection result, and notifies theillumination control unit 122 of the determined light emissionintensity.

Further, with the transmission of the determined light emissionintensity to the illumination control unit 122, the image processingunit 121 transmits the captured image that has been received from theimage capturing unit 111 to the authentication device 130, as a capturedimage for authentication.

The illumination control unit 122 evaluates the image quality of thecaptured image that has been received from the image capturing unit 111at a predetermined period, and calculates the light emission intensityof each of the sets 112_1 to 112_12 of LEDs and diffractive opticalelements which is necessary for detecting the face area of the subject.Further, the illumination control unit 122 controls each of the sets112_1 to 112_12 of LEDs and diffractive optical elements to emit lightwith the calculated light emission intensity.

In addition, in order to obtain the captured image for authentication,when the light emission intensity is notified from the image processingunit 121, the illumination control unit 122 controls the correspondingset of LED and diffractive optical element (any one of the sets 112_1 to112_12) to emit light with the notified light emission intensity.

The authentication device 130 calculates biometric information of theface area of the subject based on the captured image for authenticationthat has been transmitted from the control device 120, and compares thecalculated biometric information with the biometric information of theface area registered in advance, so as to perform the faceauthentication.

Configuration of Monitoring Device

Next, the configuration of the monitoring device 110 will be described.FIGS. 2A to 2C are views illustrating the configuration of themonitoring device. FIG. 2A illustrates an external appearanceconfiguration of the monitoring device 110. As illustrated in FIG. 2A,the monitoring device 110 includes a mounting jig 250 and is mountedoutside a building or at a high place of building entrance.

FIG. 2B illustrates a device layout around the image capturing unit 111when the monitoring device 110 is viewed from the front. As illustratedin FIG. 2B, in the case of the monitoring device 110, the set 112_1 ofLED 201_1 and diffractive optical element 202_1, the set 112_2 of LED201_2 and diffractive optical element 202_2, and the set 112_3 of LED201_3 and diffractive optical element 202_3 are arranged horizontallyfrom the left on the upper side of the image capturing unit 111 whenviewed from the front.

Although reference numerals are omitted in FIG. 2B due to spacelimitations, in the case of the monitoring device 110, the set 112_4 ofLED 201_4 and diffractive optical element 202_4, the set 112_5 of LED201_5 and diffractive optical element 202_5, and the set 112_6 of LED201_6 and diffractive optical element 202_6 are arranged vertically fromabove on the right side of the image capturing unit 111 when viewed fromthe front.

Similarly, in the case of the monitoring device 110, the set 112_7 ofLED 201_7 and diffractive optical element 202_7, the set 112_8 of LED201_8 and diffractive optical element 202_8, and the set 112_9 of LED201_9 and diffractive optical element 202_9 are arranged horizontallyfrom the right on the lower side of the image capturing unit 111 whenviewed from the front.

Similarly, in the case of the monitoring device 110, the set 112_10 ofLED 201_10 and diffractive optical element 202_10, the set 112_11 of LED201_11 and diffractive optical element 202_11, and the set 112_12 of LED201_12 and diffractive optical element 202_12 are arranged verticallyfrom below on the left side of the image capturing unit 111 when viewedfrom the front.

FIG. 2C illustrates the arrangement of each set from the set 112_1 ofLED 201_1 and diffractive optical element 202_1 to the set 112_3 of LED201_3 and diffractive optical element 202_3 when viewed from above themonitoring device 110. As illustrated in FIG. 2C, the diffractiveoptical elements 202_1 to 202_3 are arranged on a diffractive opticalelement arrangement substrate 210. The LEDs 201_1 to 201_3 are arrangedon a LED mounting substrate 211 at positions corresponding to thepositions where the diffractive optical elements 202_1 to 202_3 arearranged.

As a result, light emitted by the LED 201_1 is converted intorectangular spot light via the diffractive optical element 202_1, and isirradiated on the image capturing range of the image capturing unit 111as rectangular spot lighting. Similarly, light emitted by the LED 201_2is converted into rectangular spot light via the diffractive opticalelement 202_2, and is irradiated on the image capturing range of theimage capturing unit 111 as rectangular spot lighting. Similarly, lightemitted by the LED 201_3 is converted into rectangular spot light viathe diffractive optical element 202_3, and is irradiated on the imagecapturing range of the image capturing unit 111 as rectangular spotlighting.

It is assumed that the sets 112_4 to 112_12 are arranged in the samemanner as in FIG. 2C and their respective LEDs, and diffractive opticalelements are arranged in diffractive optical element arrangementsubstrates 220, 230, and 240 and LED mounting substrates 221, 231, and241, respectively.

Example of Configuration of Diffractive Optical Element

Next, the configuration of a diffractive optical element (here, thediffractive optical element 202_1) will be described. FIGS. 3A and 3Bare views illustrating an example of a configuration of a diffractiveoptical element. As illustrated in FIG. 3A, the diffractive opticalelement 202_1 is formed by arranging linear diffraction gratings (cells)two-dimensionally in the same plane. The example in FIG. 3A represents acase where 50 diffraction gratings (cells) of 0.02 mm×0.02 mm arearranged in the vertical direction and 50 diffraction gratings of 0.02mm×0.02 mm are arranged in the horizontal direction (250 in total).

Each of the diffraction gratings generates a plurality of dot lightswhen the light emitted from the LED 201_1 is transmitted. In addition,since the respective diffraction gratings have different pitches androtation angles, the generated dot lights are combined in the imagecapturing range of the image capturing unit 111. Accordingly, the set112_1 of LED 201_1 and diffractive optical element 202_1 may irradiatethe image capturing range of the image capturing unit 111 with uniformrectangular spot lighting.

FIG. 3B is a view illustrating the correspondence relationship betweenthe image capturing range of the image capturing unit 111 which isirradiated by each of the sets of LEDs and diffractive optical elementswith rectangular spot lighting, and the captured image of the imagecapturing unit 111. As illustrated in FIG. 3B, a captured image 300 ofthe image capturing unit 111 is divided into a plurality of blocksaccording to the number of sets of LEDs and diffractive opticalelements, and the corresponding set of LED and diffractive opticalelement irradiates the image capturing range corresponding to each blockwith rectangular spot lighting.

As described above, since 12 sets of LEDs and diffractive opticalelements are arranged around the image capturing unit 111, the capturedimage 300 of the image capturing unit 111 is divided into 12 blocks. InFIG. 3B, each block is numbered, and the corresponding set of LED anddiffractive optical element is indicated by a lead line.

For example, the set 112_1 of LED and diffractive optical elementirradiates the image capturing range corresponding to the block of blocknumber=“1” with rectangular spot lighting.

Hardware Configuration of Control Device

Next, the hardware configuration of the control device 120 will bedescribed. FIG. 4 is a view illustrating an example of the hardwareconfiguration of the control device.

As illustrated in FIG. 4 , the control device 120 includes a CPU(Central Processing Unit) 401, a ROM (Read Only Memory) 402, and a RAM(Random Access Memory) 403. The CPU 401, the ROM 402, and the RAM 403make up a so-called computer. Further, the control device 120 includesan I/F (Interface) device 404, an illumination control device 405, and acommunication device 406. The respective units of the control device 120are interconnected via a bus 407.

The CPU 401 is an arithmetic device that executes various programs(e.g., a control program and the like) installed in the ROM 402. The ROM402 is a nonvolatile memory, and functions as a main memory device thatstores the various programs executed by the CPU 401 and information usedwhen the CPU 401 executes the various programs.

The RAM 403 is a volatile memory such as a DRAM (Dynamic Random AccessMemory), a SRAM (Static Random Access Memory) or the like. The RAM 403functions as a main memory device that provides a work area that isdeployed when the various programs installed in the ROM 402 are executedby the CPU 401.

The I/F device 404 is a connection device for connecting to the imagecapturing unit 111 of the monitoring device 110. The illuminationcontrol device 405 is a control device for controlling the LED mountingsubstrates 211, 221, 231, and 241 of the monitoring device 110. Thecommunication device 406 communicates with the authentication device130.

Functional Configuration of Control Device

Next, the functional configuration of the control device 120 will bedescribed. FIG. 5 is a first view illustrating an example of thefunctional configuration of the control device. As illustrated in FIG. 5, the image processing unit 121 includes an image acquisition unit 501,a face detection unit 502, a face position detection unit 503, and alight emission intensity control unit 504.

The image acquisition unit 501 acquires the captured image that has beentransmitted from the image capturing unit 111 of the monitoring device110, and notifies the captured image to the face detection unit 502. Theimage acquisition unit 501 also transmits the captured image that hasbeen transmitted from the image capturing unit 111 of the monitoringdevice 110, to the authentication device 130 as a captured image forauthentication.

The face detection unit 502 is an example of a detection unit, andprocesses the captured image that has been notified from the imageacquisition unit 501 to detect the face area of the subject. The facedetection unit 502 also notifies the face position detection unit 503 ofthe detected face area. Further, the face detection unit 502 determinesa block including the detected face area, and notifies the block to thelight emission intensity output unit 507.

The face position detection unit 503 is an example of a determinationunit, and detects a position of the face area of the subject notifiedfrom the face detection unit 502, that is, a predetermined positionwithin the face area, for example, the center position of the face area,the position of a specific part (eye, nose, etc.), or the like, anddetermines a block to which the detected position of the face areabelongs. Further, the face position detection unit 503 notifies thelight emission intensity control unit 504 of the block to which theposition of the face area belongs.

The light emission intensity control unit 504 is an example of a controlunit, and based on the block notified from the face position detectionunit 503, determines the light emission intensity of a set of LED anddiffractive optical element that irradiates the image capturing rangecorresponding to the block with spot lighting.

As illustrated in a table 510 of FIG. 5 , it is assumed that thecorrespondence relationship between a block to which the position of theface area of the subject belongs and the light emission intensity isdefined in advance. This is because the image capturing distance to thesubject may be roughly estimated based on a block of the captured imageto which the face area of the subject belongs, and the light emissionintensity may be defined according to the estimated image capturingdistance.

The example of the table 510 in FIG. 5 represents that the blocks of thecaptured image are divided into four groups, and different lightemission intensities are associated with the groups, respectively.

Further, as illustrated in FIG. 5 , the illumination control unit 122includes an image quality evaluation unit 505, an adjustment unit 506,and a light emission intensity output unit 507.

The image quality evaluation unit 505 acquires the captured image thathas been transmitted from the image capturing unit 111 of the monitoringdevice 110, and evaluates the quality of the acquired captured image.Specifically, the image quality evaluation unit 505 divides the acquiredcaptured image into a plurality of blocks, and evaluates whether or noteach block has a level of image quality where a face area may bedetected. The image quality evaluation unit 505 also notifies theadjustment unit 506 of the evaluation result for each block.

The adjustment unit 506 adjusts the light emission intensity of the setof LED and diffractive optical element that irradiate the imagecapturing range corresponding to each block with spot lighting accordingto the evaluation result for each block notified from the image qualityevaluation unit 505. Further, the adjustment unit 506 notifies the lightemission intensity output unit 507 of the adjusted light emissionintensity. As a result, the image capturing unit 111 may obtain acaptured image that has a level of image quality where a face area maybe detected.

The image quality evaluation unit 505 and the adjustment unit 506operate at a predetermined period, for example, each time thesurrounding environment changes, and therefore, the image capturing unit111 may obtain a captured image that has a level of image quality wherea face area may be detected even when the surrounding environmentchanges.

When the light emission intensity corresponding to each block isnotified from the adjustment unit 506, the light emission intensityoutput unit 507 transmits the notified light emission intensity to theLED mounting substrates 211, 221, 231, and 241. As a result, each set ofLED and diffractive optical element that irradiates the image capturingrange that corresponds to each block with spot lighting may emit lightwith the notified light emission intensity.

In addition, when the light emission intensity is notified from thelight emission intensity control unit 504, the light emission intensityoutput unit 507 transmits the notified light emission intensity to anyof the LED mounting substrates 211, 221, 231, and 241. As a result, theset of LED and diffractive optical element that irradiates the imagecapturing range that corresponds to the block notified from the facedetection unit 502 with spot lighting may emit light with the notifiedlight emission intensity.

As a result, the image capturing unit 111 may obtain a captured imagewith the face area irradiated with spot lighting with the light emissionintensity according to the image capturing distance to the subject.

Example of Operation of Monitoring System

Next, an example of the operation of the monitoring system 100 will bedescribed. FIGS. 6A and 6B are first views illustrating an example ofthe operation of the monitoring system. FIG. 6A illustrates a statewhere a subject 610 is approaching the monitoring device 110.

FIG. 6B illustrates captured images 631, 632, and 633 that are obtainedby the monitoring device 110 at respective positions (positions 621,622, and 623) of the subject 610 illustrated in FIG. 6A. It is assumedthat when the captured images 631, 632, and 633 are obtained, theadjustment of the light emission intensity by the adjustment unit 506has been completed, and the captured images 631, 632, and 633 have alevel of image quality where the face area of the subject 610 may bedetected.

In the case of the captured image 631, the face detection unit 502determines that the block number of the block including the face area ofthe subject 610 is “2.” Further, the face position detection unit 503determines that the block number of the block to which the detectedposition of the face area of the subject 610 belongs is “2.”

Accordingly, the light emission intensity control unit 504 determinesthe light emission intensity to be “I4.” Then, the light emissionintensity output unit 507 controls the set 112_2 of LED 201_2 anddiffractive optical element 202_2 to emit light with the light emissionintensity “I4.”

Similarly, in the case of the captured image 632, the face detectionunit 502 determines that the block number of the block including theface area of the subject 610 is “5.” Further, the face positiondetection unit 503 determines that the block number of the block towhich the detected position of the face area of the subject 610 belongsis “5.”

Accordingly, the light emission intensity control unit 504 determinesthe light emission intensity to be “I3.” Then, the light emissionintensity output unit 507 controls the set 112_12 of LED 201_12 anddiffractive optical element 202_12 to emit light with the light emissionintensity “I3.”

Similarly, in the case of the captured image 633, the face detectionunit 502 determines that the block number of the block including theface area of the subject 610 is “8, 11.” Further, the face positiondetection unit 503 determines that the block number of the block towhich the detected position of the face area of the subject 610 belongsis “I1.”

Accordingly, the light emission intensity control unit 504 determinesthe light emission intensity to be “I1”. Then, the light emissionintensity output unit 507 controls the set 112_6 of LED 201_6 anddiffractive optical element 202_6 and the set 112_8 of LED 201_8 anddiffractive optical element 202_8 to emit light with the light emissionintensity “I1.”

Flow of Control Process

Next, a flow of the control process by the control device 120 will bedescribed. FIG. 7 is a first flowchart illustrating a flow of thecontrol process by the control device. In operation S701, the imageacquisition unit 501 acquires a captured image from the image capturingunit 111 of the monitoring device 110.

In operation S702, the image quality evaluation unit 505 determineswhether or not each block of the acquired captured image has a level ofimage quality where the face area of the subject may be detected(whether the image quality is equal to or higher than a predeterminedlevel).

When it is determined in operation S702 that the image quality of anyblock of the captured image is lower than the predetermined level (No inoperation S702), the process proceeds to operation S703.

In operation S703, the adjustment unit 506 adjusts the light emissionintensity of a set of LED and diffractive optical element thatcorresponds to the block whose image quality is determined to be lowerthan the predetermined level, and the process returns to operation S701.

Meanwhile, when it is determined in operation S702 that the imagequality of each block of the acquired captured image is equal to orhigher than the predetermined level (Yes in operation S702), the processproceeds to operation S704. It is assumed that the processes ofoperation operations S702 and S703 are executed every predeterminedperiod.

In operation S704, the face detection unit 502 processes the acquiredcaptured image to detect the face area of the subject.

In operation S705, the face position detection unit 503 determines ablock to which the position of the detected face area of the subjectbelongs.

In operation S706, the light emission intensity control unit 504determines the light emission intensity based on the block to which theposition of the face area of the subject belongs.

In operation S707, the face detection unit 502 determines a blockincluding the detected face area. Thereby, the light emission intensitycontrol unit 504 selects a set of LED and diffractive optical element tobe controlled to emit light with the determined light emissionintensity.

In operation S708, the light emission intensity output unit 507transmits the determined light emission intensity to the LED mountingsubstrate so that the selected set of LED and diffractive opticalelement emits light with the determined light emission intensity.

In operation S709, the image acquisition unit 501 acquires a capturedimage from the image capturing unit 111 of the monitoring device 110 andtransmits the acquired captured image to the authentication device 130as a captured image for authentication.

As is apparent from the above description, the monitoring systemaccording to the first embodiment detects the face area of the subjectfrom the captured image of the image capturing unit, which is dividedinto a plurality of blocks according to the number of LEDs thatirradiate the image capturing range of the image capturing unit.Further, the monitoring system according to the first embodimentcontrols an LED corresponding to a block including the face area of thesubject among the plurality of blocks to emit light with the lightemission intensity predetermined according to a block to which theposition of the face area of the subject belongs.

In this way, with the monitoring system according to the firstembodiment, by controlling the light emission intensity using therelationship between the block to which the position of the face area ofthe subject belongs and the image capturing distance to the subject, itis possible to capture the subject's face with constant brightness,regardless of the image capturing distance to the subject.

In addition, the monitoring system according to the first embodimentadjusts the light emission intensity every predetermined period so as toobtain a captured image with an image quality enough to detect the facearea.

As a result, with the monitoring system according to the firstembodiment, it is possible to detect the face area to capture the faceof the subject with constant brightness even when the surroundingenvironment changes.

Second Embodiment

The case where the light emission intensity is determined based on theblock to which the position of the face area of the subject belongs hasbeen described in the first embodiment. However, the method fordetermining the light emission intensity is not limited thereto, but thelight emission intensity may be determined according to the size of theface area of the subject. Hereinafter, a second embodiment will bedescribed focusing on the differences from the first embodiment.

Functional Configuration of Control Device

First, the functional configuration of the control device 120 accordingto the second embodiment will be described. FIG. 8 is a second viewillustrating an example of the functional configuration of the controldevice. A difference from FIG. 5 lies in a face size calculation unit801 and a light emission intensity control unit 802.

The face size calculation unit 801 is an example of a calculation unit,and calculates the size of the face area of the subject notified fromthe face detection unit 502. Further, the face size calculation unit 801notifies the light emission intensity control unit 802 of the calculatedsize of the face area.

The light emission intensity control unit 802 is an example of a controlunit, and determines the light emission intensity based on the size ofthe face area notified from the face size calculation unit 801. Inaddition, as illustrated in the table 810 of FIG. 8 , it is assumed thatthe correspondence relationship between the size of the face area of thesubject and the light emission intensity is defined in advance. This isbecause the image capturing distance to the subject may be roughlyestimated according to the size of the face area of the subject, and thelight emission intensity may be defined according to the estimated imagecapturing distance.

The example of the table 810 in FIG. 8 represents that the size of theface area is divided into five groups, and different light emissionintensities are associated with the groups, respectively. However,instead of the table 810, the correspondence relationship between thesize of the face area and the light emission intensity may be defined bya predetermined function.

Example of Operation of Monitoring System

Next, an example of the operation of the monitoring system 100 will bedescribed. FIGS. 9A and 9B are second views illustrating an example ofthe operation of the monitoring system. FIG. 9A illustrates a state inwhich the subject 610 is approaching the monitoring device 110.

FIG. 9B illustrates captured images 631, 902, and 633 captured by themonitoring device 110 at respective positions (positions 621, 901, and623) of the subject 610 illustrated in FIG. 9A. It is assumed that whenthe captured images 631, 902, and 633 are obtained, the adjustment ofthe light emission intensity by the adjustment unit 506 has beencompleted, and the captured images 631, 902, and 633 have a level ofimage quality where the face area of the subject 610 may be detected.

In the case of the captured image 631, the face detection unit 502determines that the block number of the block including the face area ofthe subject 610 is “2.” Further, the face size calculation unit 801determines that the size of the face area of the subject 610 is “SS.”

Accordingly, the light emission intensity control unit 802 determinesthe light emission intensity to be “I5.” Then, the light emissionintensity output unit 507 controls the set 112_2 of LED 201_2 anddiffractive optical element 202_2 to emit light with the light emissionintensity “I5.”

Similarly, in the case of the captured image 902, the face detectionunit 502 determines that the block number of the block including theface area of the subject 610 is “2.” Further, the face size calculationunit 801 determines that the size of the face area of the subject 610 is“S.”

Accordingly, the light emission intensity control unit 802 determinesthe light emission intensity to be “I4”. Then, the light emissionintensity output unit 507 controls the set 112_2 of LED 201_2 anddiffractive optical element 202_2 to emit light with the light emissionintensity “I4.”

Similarly, in the case of the captured image 633, the face detectionunit 502 determines that the block number of the block including theface area of the subject 610 is “8, 11.” The face size calculation unit801 determines that the size of the face area of the subject 610 is “L.”

Accordingly, the light emission intensity control unit 802 determinesthe light emission intensity to be “I2.” Then, the light emissionintensity output unit 507 controls the set 112_6 of LED 201_6 anddiffractive optical element 202_6 and the set 112_8 of LED 201_8 anddiffractive optical element 202_8 to emit light with the light emissionintensity “I2.”

Flow of Control Process

Next, a flow of the control process by the control device 120 accordingto the second embodiment will be described. FIG. 10 is a secondflowchart illustrating a flow of the control process by the controldevice. The difference from the first flow chart illustrated in FIG. 7lies in operations S1001 and S1002.

In operation S1001, the face size calculation unit 801 calculates thesize of the face area of the subject notified from the face detectionunit 502.

In operation S1002, the light emission intensity control unit 802determines the light emission intensity based on the size of the facearea calculated by the face size calculation unit 801.

As is apparent from the above description, the monitoring systemaccording to the second embodiment detects the face area of the subjectfrom the captured image of the image capturing unit, which is dividedinto a plurality of blocks according to the number of LEDs thatirradiate the image capturing range of the image capturing unit.Further, the monitoring system according to the second embodimentcontrols an LED corresponding to a block including the face area of thesubject among the plurality of blocks to emit light with the lightemission intensity predetermined according to the size of the face areaof the subject.

In this way, with the monitoring system according to the secondembodiment, by controlling the light emission intensity using therelationship between the size of the face area of the subject and theimage capturing distance to the subject, it is possible to capture thesubject's face with constant brightness, regardless of the imagecapturing distance to the subject.

Third Embodiment

The case where the light emission intensity is determined based on theblock to which the position of the face area of the subject belongs hasbeen described in the first embodiment. However, the method fordetermining the light emission intensity is not limited thereto, but thelight emission intensity may be determined according to the imagequality of the block including the face area of the subject.Hereinafter, a third embodiment will be described focusing on thedifferences from the first embodiment.

Functional Configuration of Control Device

First, the functional configuration of the control device 120 accordingto the third embodiment will be described. FIG. 11 is a third viewillustrating an example of the functional configuration of the controldevice. The difference from FIG. 5 is a block image quality evaluationunit 1101 and a light emission intensity control unit 1102.

The block image quality evaluation unit 1101 is an example of anevaluation unit, and evaluates the image quality of a block includingthe face area of the subject notified from the face detection unit 502.Specifically, the block image quality evaluation unit 1101 evaluateswhether or not the block including the face area of the subject has alevel of image quality where the authentication device 130 may execute aface authentication process. Further, the block image quality evaluationunit 1101 notifies the light emission intensity control unit 1102 of theevaluation result.

The light emission intensity control unit 1102 is an example of acontrol unit, and determines the light emission intensity based on theevaluation result notified from the block image quality evaluation unit1101. Specifically, the light emission intensity control unit 1102determines the light emission intensity of a set of LED and diffractiveoptical element that irradiates the corresponding image capturing rangewith spot lighting, so that the authentication device 130 may obtain animage quality necessary to execute the face authentication process.

Flow of Control Process

Next, a flow of the control process by the control device 120 accordingto the third embodiment will be described. FIG. 12 is a third flowchartillustrating a flow of the control process by the control device. Thedifference from the first flow chart illustrated in FIG. 7 lies inoperations S1201 to S1205.

In operation S1201, the block image quality evaluation unit 1101evaluates the image quality of a block including the face area of thesubject notified from the face detection unit 502.

In operation S1202, the block image quality evaluation unit 1101determines whether or not each block including the face area has a levelof image quality where the face authentication process may be executed(whether the image quality is in a level where a face authentication ispossible). When it is determined in operation S1202 that the block doesnot have a level of image quality where the face authentication ispossible (No in operation S1202), the process proceeds to operationS1203.

In operation S1203, the light emission intensity control unit 1102determines the light emission intensity based on the evaluation result.

In operation S1204, the light emission intensity output unit 507transmits the determined light emission intensity to the LED mountingsubstrate so that a set of LED and diffractive optical element thatirradiates the image capturing range corresponding to each blockincluding the face area may emit light with the determined lightemission intensity.

In operation S1205, the image acquisition unit 501 acquires the capturedimage from the monitoring device 110, and the process returns tooperation S704

Meanwhile, when it is determined in operation S1202 that the imagequality is in a level where the face authentication is possible (Yes inoperation S1202), the process proceeds to operation S709.

In operation S709, the image acquisition unit 501 acquires the capturedimage from the image capturing unit 111 of the monitoring device 110 andtransmits the acquired captured image to the authentication device 130as a captured image for authentication.

As is apparent from the above description, the monitoring systemaccording to the third embodiment detects the face area of the subjectfrom the captured image of the image capturing unit, which is dividedinto a plurality of blocks according to the number of LEDs thatirradiate the image capturing range of the image capturing unit.Further, the monitoring system according to the third embodimentcontrols an LED corresponding to a block including the face area of thesubject among the plurality of blocks to emit light with the lightemission intensity according to the evaluation result of the imagequality of the block including the face area of the subject.

In this way, with the monitoring system according to the thirdembodiment, by controlling the light emission intensity using theevaluation result of the image quality of the block in which the facearea of the subject is captured, it is possible to capture the subject'sface with constant brightness, regardless of the image capturingdistance to the subject and the surroundings.

Fourth Embodiment

The case where there is one subject appearing in the captured imagetransmitted from the image capturing unit 111 has been described in thefirst embodiment. However, the number of subjects appearing in thecaptured image transmitted from the image capturing unit 111 is notlimited to one. In a fourth embodiment, a case will be described inwhich a plurality of subjects appear in the captured image transmittedfrom the image capturing unit 111. The description will be focused onthe differences from the first embodiment.

Functional Configuration of Control Device

First, the functional configuration of the control device 120 accordingto the fourth embodiment will be described. FIG. 13 is a fourth viewillustrating an example of the functional configuration of the controldevice. The difference from FIG. 5 lies in a face area number detectionunit 1301.

The face area number detection unit 1301 counts the number of subject'sface areas notified from the face detection unit 502, and notifies theface position detection unit 503 of the counted number of subject's faceareas together with the face areas. Thereby, the face position detectionunit 503 detects face area positions for the subject's face areas thatcorrespond to the notified number of subject's face areas, anddetermines a block to which each detected face area position belongs.Further, the light emission intensity control unit 504 refers to thetable 510 to determine the light emission intensity of a set of LED anddiffractive optical element that irradiates the image capturing rangecorresponding to each block with spot lighting.

Example of Operation of Monitoring System

Next, an example of the operation of the monitoring system 100 includingthe control device 120 according to the fourth embodiment will bedescribed. FIGS. 14A and 14B are third views illustrating an example ofthe operation of the monitoring system. FIG. 14A illustrates a state inwhich the subject 610 is approaching the monitoring device 110.

While FIG. 14A does not illustrate subjects other than the subject 610due to space limitations, it is assumed that, at the point of time whenthe subject 610 is walking at the position 621, another subject 1410 iswalking at a position closer to the monitoring device 110 than thesubject 610. Further, it is assumed that, at the point of time when thesubject 610 is walking at the position 622, another subject 1410 iswalking at a position closer to the monitoring device 110 than thesubject 610. Furthermore, it is assumed that, at the point of time whenthe subject 610 is walking at the position 623, another subject 1420 iswalking at a position farther from the monitoring device 110 than thesubject 610.

FIG. 14B illustrates captured images 1401, 1402, and 1403 obtained bythe monitoring device 110 at respective positions (positions 621, 622,and 623) of the subject 610 illustrated in FIG. 14A. It is assumed thatwhen the captured images 1401, 1402, and 1403 are obtained, theadjustment of the light emission intensity by the adjustment unit 506has been completed, and the captured images 1401, 1402, and 1403 have alevel of image quality where the face areas of the subjects 610, 1410,and 1420 may be detected.

In the case of the captured image 1401, the face detection unit 502determines that the block numbers of the blocks including the face areasof the subjects 610 and 1410 are “2” and “6.” Further, the face areanumber detection unit 1301 determines that the number of face areas is“2.” Further, the face position detection unit 503 determines that theblock number of the block to which the detected face area position ofthe subject 610 belongs is “2,” and the block number of the block towhich the detected face area position of the subject 1410 belongs is“6.”

Accordingly, the light emission intensity control unit 504 determinesthe light emission intensity to be “I4” or “I3.” Then, the lightemission intensity output unit 507 controls the set 112_2 of LED 201_2and diffractive optical element 202_2 to emit light with the lightemission intensity “I4,” and controls the set 112_4 of LED 201_4 anddiffractive optical element 202_4 to emit light with the light emissionintensity “I3.”

Similarly, in the case of the captured image 1402, the face detectionunit 502 determines that the block numbers of the blocks including theface areas of the subjects 610 and 1410 are “5” and “9, 12.” Further,the face area number detection unit 1301 determines that the number offace areas is “2.” Further, the face position detection unit 503determines that the block number of the block to which the detected facearea position of the subject 610 belongs is “5,” and the block number ofthe block to which the detected face area position of the subject 1410belongs is “9.”

Accordingly, the light emission intensity control unit 504 determinesthe light emission intensity to be “I3” or “I2.” Then, the lightemission intensity output unit 507 controls the set 112_12 of LED 201_12and diffractive optical element 202_12 to emit light with the lightemission intensity “I3.” Further, the light emission intensity outputunit 507 controls the set 112_5 of LED 201_5 and diffractive opticalelement 202_5 and the set 112_7 of LED 201_7 and diffractive opticalelement 202_7 to emit light with the light emission intensity “I2.”

Similarly, in the case of the captured image 1403, the face detectionunit 502 determines that the block numbers of the blocks including theface areas of the subjects 610 and 1420 are “8, 11” and “1.” Further,the face area number detection unit 1301 determines that the number offace areas is “2.” Further, the face position detection unit 503determines that the block number of the block to which the detected facearea position of the subject 610 belongs is “11,” and the block numberof the block to which the detected face area position of the subject1420 belongs is “1.”

Accordingly, the light emission intensity control unit 504 determinesthe light emission intensity to be “I1” or “I4.” Then, the lightemission intensity output unit 507 controls the set 112_6 of LED 201_6and diffractive optical element 202_6 and the set 112_8 of LED 201_8 anddiffractive optical element 202_8 to emit light with the light emissionintensity “IL” Further, the light emission intensity output unit 507controls the set 112_1 of LED 201_1 and diffractive optical element202_1 to emit light with the light emission intensity “I4.”

Flow of Control Process

Next, a flow of the control process by the control device 120 accordingto the fourth embodiment will be described. FIG. 15 is a fourth flowchart illustrating a flow of the control process by the control device.The difference from the flowchart illustrated in FIG. 7 lies inoperation S1501.

In operation S1501, the face area number detection unit 1301 counts thenumber of subject's face areas detected in operation S704. Thereby, inoperation S705, the face position detection unit 503 detects face areapositions for the notified number of subject's face areas, anddetermines a block to which each detected face area position belongs. Inoperation S706, the light emission intensity control unit 504 determinesthe light emission intensity corresponding to each block. Further, inoperation S707, the face detection unit 502 determines a block includingeach of the detected face areas, and the light emission intensitycontrol unit 504 selects a set of LED and diffractive optical element,which controls to emit light with the determined emission intensity.

As is apparent from the above description, the monitoring systemaccording to the fourth embodiment performs the same process as in thefirst embodiment on the face area of each of a plurality of face areasof the subjects appearing in the captured image transmitted from theimage capturing unit 111.

As a result, the monitoring system according to the fourth embodimentmay obtain the same effects as the first embodiment.

OTHER EMBODIMENTS

In the above-described first and second embodiments, the light emissionintensity control units 504 and 802 have been described as having thetables 510 and 810, respectively. However, each of the light emissionintensity control units 504 and 802 may have a plurality of tables 510and 810 in order to cope with a change in the surrounding environmentsuch as the weather, the time zone and the like. Alternatively, insteadof the tables 510 and 810, the light emission intensity control units504 and 802 may perform a machine learning in advance on thecorrespondence relationship between the block to which the face areaposition belongs along with the surrounding environment, and the lightemission intensity of the set of LED and diffractive optical element,and may use the learning result at that time.

In addition, in the control device 120, for example, the image qualityevaluation unit 505 evaluates the quality of the captured image tospecify the surrounding environment. Further, the light emissionintensity control unit 504 determines the light emission intensity byreferring to the table corresponding to the surrounding environmentspecified by the image quality evaluation unit 505, among the pluralityof tables. Alternatively, the light emission intensity control unit 504may determine the light emission intensity by inputting the block towhich the subject' face area position belongs and the specifiedsurrounding environment into the learning result.

In addition, in the third embodiment, the correspondence relationshipbetween the image quality of the block including the face area and thelight emission intensity of a set of LED and diffractive optical elementis not mentioned, but the correspondence relationship may be, forexample, defined by a table. Alternatively, the correspondencerelationship may be defined based on machine learning.

In addition, in each of the above-described embodiments, the monitoringdevice 110 in which the image capturing unit 111 and the sets 112_1 to112_12 of LEDs and diffractive optical elements are integrated has beendescribed. However, the image capturing unit 111 and the set 112_1 to112_12 of LEDs and diffractive optical elements may be separate devices.

In addition, in each of the above-described embodiments, the monitoringdevice 110 and the control device 120 are separate devices. However, themonitoring device 110 and the control device 120 may be an integrateddevice. Alternatively, some of the functions of the control device 120may be implemented in the monitoring device 110.

The present disclosure is not limited to the elements described herein,and for example, the elements described in the foregoing embodiments maybe combined with other elements. In view of this point, the presentdisclosure may be changed without departing from the gist of the presentdisclosure, and may be appropriately determined according to theapplication form.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the inventionand the concepts contributed by the inventor to furthering the art, andare to be construed as being without limitation to such specificallyrecited examples and conditions, nor does the organization of suchexamples in the specification relate to an illustrating of thesuperiority and inferiority of the invention. Although the embodimentsof the present invention have been described in detail, it should beunderstood that the various changes, substitutions, and alterationscould be made hereto without departing from the spirit and scope of theinvention.

What is claimed is:
 1. A monitoring system receiving, from an imagecapturing sensor, a captured image of an image capturing rangeilluminated by light emitting elements, comprising: a memory; and aprocessor coupled to the memory and configured to: detect a face area ofa subject from the captured image in the image capturing range dividedinto a rectangular array formed by rows and columns of a plurality ofblocks according to the light emitting elements, formed in four lineararrays of at least three light emitting elements each, disposedalongside and substantially parallel to each of four sides of the imagesensor, the light emitting elements having corresponding sets ofdiffractive optical elements that each convert light emitted from thelight emitting elements into rectangular spot light that irradiate theimage capturing range with each block irradiated by the rectangular spotlight formed by one of the diffractive optical elements; and control alight emission intensity of a light emitting element corresponding to ablock including the face area among the plurality of blocks according toone of a position of the face area and a size of the face area.
 2. Themonitoring system according to claim 1, wherein the blocks correspond torectangles of spot light formed by diffractive optical elements throughwhich light from the light emitting elements pass, respectively, whereinthe processor is further configured to determine the block to which theposition of the face area belongs, and wherein the processor isconfigured to control the light emission intensity passing through thediffractive optical elements by referring to a table in which the lightemission intensity is associated in advance for each block to which theposition of the face area belongs.
 3. The monitoring system according toclaim 1, wherein the blocks correspond to rectangles of spot lightformed by diffractive optical elements through which light from thelight emitting elements pass, respectively, wherein the processor isfurther configured to calculate the size of the face area, and whereinthe processor is configured to control the light emission intensitypassing through the diffractive optical elements by referring to a tablein which the light emission intensity is associated in advance for eachsize of the face area.
 4. The monitoring system according to claim 1,wherein the blocks correspond to rectangles of spot light formed bydiffractive optical elements through which light from the light emittingelements pass, respectively, wherein the processor is further configuredto count a number of face areas, and wherein the processor is configuredto control the light emission intensity of the light passing througheach of the diffractive optical elements corresponding to each of theblocks included in the face areas according to at least one of theposition and the size of each of the face areas of the counted number.5. The monitoring system according to claim 1, wherein an image qualityof the captured image is evaluated every predetermined interval, and thelight emission intensity is adjusted based on the evaluation result sothat the face area of the subject is detected.
 6. The monitoring systemaccording to claim 1, wherein the four linear arrays of the at leastthree light emitting elements and the corresponding sets of thediffractive optical elements are integrated with the image capturingsensor that generates the captured image.
 7. A monitoring systemreceiving, from an image capturing sensor, a captured image of an imagecapturing range illuminated by light emitting elements, comprising: amemory; and a processor coupled to the memory and configured to: detecta face area of a subject from a captured image in the image capturingrange divided into a rectangular array formed by rows and columns of aplurality of blocks according to the light emitting elements, formed infour linear arrays of at least three light emitting elements each,disposed alongside and substantially parallel to each of four sides ofthe image sensor, the light emitting elements having corresponding setsof diffractive optical elements that each convert light emitted from thelight emitting elements into rectangular spot light that irradiate theimage capturing range with each block irradiated by the rectangular spotlight formed by one of the diffractive optical elements; and control alight emission intensity of a light emitting element corresponding to ablock including the face area among the plurality of blocks according toan image quality of the block.
 8. A monitoring method comprising:detecting a face area of a subject from a captured image of an imagecapturing range divided into a rectangular array formed by rows andcolumns of a plurality of blocks according to light emitting elements,formed in four linear arrays of at least three light emitting elementseach, disposed alongside and substantially parallel to each of fourssides of an image sensor, the light emitting elements havingcorresponding sets of diffractive optical elements that each convertlight emitted from the light emitting elements into rectangular spotlight that irradiate the image capturing range with each blockirradiated by the rectangular spot light formed by one of thediffractive optical elements; and controlling a light emission intensityof a light emitting element corresponding to a block including the facearea among the plurality of blocks according to one of a position of theface area and a size of the face area, by a processor.